STT and TTS corrections

The agent can correct the text on both sides of the Large Language Model: the user's utterance as the STT engine wrote it, before it reaches the model, and the model's response, before it is sent to the TTS engine.

These corrections apply to Agents and Flows on regular (non-speech-to-speech) models. Speech-to-speech models take in the user's audio and produce their spoken response themselves, with no STT or TTS text in between to correct.

STT corrections

The remove_symbols and replace_words advanced configuration parameters correct Speech-to-Text (STT) errors in user input, helping to improve the accuracy of voice interactions. They run in that order, followed by numbers_sequence.

When a correction changes the utterance, the conversation log shows a USER INPUT AFTER CORRECTION entry with the text the model received.

Replacing misheard words

Use the replace_words parameter to automatically correct commonly misidentified words in user speech.

Parameter Type Description
replace_words dict[str, str] JSON dictionary mapping incorrect words to their correct replacements.

The specified words are matched case-sensitively and at word boundaries.

Example
{
    "replace_words": {"halo": "hello", "hye": "hi"}
}

Removing symbols

Use the remove_symbols parameter to automatically strip unwanted symbols – e.g. punctuation marks – from user input.

Parameter Type Description
remove_symbols list[str] List of symbols to be removed from user utterance.
Example
{
    "remove_symbols": [".", ",", "!"]
}

TTS corrections

The llm_replace_words, llm_allowed_alphabets and llm_add_period advanced configuration parameters modify the LLM response before it is played. This may be used, for example, to overcome Text-to-Speech (TTS) engine pronunciation errors by replacing specific words with their phonemes.

Replacing mispronounced words

Use the llm_replace_words parameter to replace specific words in LLM response (for example with their phonemes representation).

Parameter Type Description
llm_replace_words dict[str, str] JSON dictionary mapping specific words to their replacements.

The specified words are matched case-sensitively and at word boundaries.

Example
{
    "llm_replace_words": {"LiveHub": " laɪv hʌb"}
}

Restricting the alphabets

Occasionally a model ends an otherwise correct response with a few characters from an unrelated alphabet - a Spanish assistant finishing a sentence with a Devanagari word - which the TTS engine then reads out as noise. Use the llm_allowed_alphabets parameter to list the alphabets the response may be written in; characters from any other alphabet are removed before the response is played.

Parameter Type Description
llm_allowed_alphabets list[str] Alphabets allowed in the LLM response.

The supported values are latin, cyrillic, greek, hebrew, arabic, chinese, japanese, korean, thai and devanagari, plus emoji.

Digits, punctuation, whitespace, currency symbols and accents are always kept, whichever alphabets are listed. Emoji and other symbols are removed unless emoji is listed as well.

Note: List every alphabet the agent legitimately uses. Most agents need latin even when they speak another language, because email addresses, URLs, product codes and brand names are written in it - an agent configured with hebrew alone would strip the address out of "support@company.com".

If the parameter is absent or its list is empty, no filtering is performed.

Example
{
    "llm_allowed_alphabets": ["latin", "hebrew"]
}

Fixing punctuation

Use the llm_add_period parameter to ensure that every LLM response is treated as a complete sentence. If the model generates a response without end of sentence punctuation, a period will be automatically appended.

Parameter Type Description
llm_add_period bool Add period to LLM response if it lacks end-of-sentence punctuation.
Example
{
    "llm_add_period": true
}